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1. Introduction 


This report documents work performed by Boeing Air Traffic Management for the 
NASA Ames Research Center. The work was contracted through the Federal Aviation 
Administration’s (FAA’s) Office of System Capacity & Requirements support services 
contract with ATAC Corporation. The purpose of this contract work was to perform a 
preliminary evaluation of NASA’s proposed Aviation System Technology Advanced 
Research Program (AvSTAR) during the early stages of its definition, in the first half of 
the year 2001. 

The scope of work specified by the NASA Ames AvSTAR program office was as 
follows: 

NASA's Ames Research Center has proposed a new initiative/program in Air 
Traffic Management (ATM) called the Aviation System Technology Advanced 
Research Program (AvSTAR). AvSTAR’s goal is to provide research and 
development by the year 2007 that will: (1) complete the development of 
technology needed to enable Free-Flight by 2010 and (2) provide the basic R&D 
foundation to set the direction for ATM as a high-capacity, integrated component 
of a multi-modal transportation system of the future (circa 2025). AvSTAR has an 
aggressive list of quantifiable objectives/metrics for both of the target time frames 
(2010 and 2025). 

The Subcontractor shall perform an independent review of the current AvSTAR 
proposal to provide a risk assessment of the proposed program content against the 
AvSTAR system performance goals. Recommendations on how the AvSTAR 
program could be strengthened and improved shall also be provided. The review 
shall be based on expert knowledge of the current and near term ATM system and 
a high level assessment to estimate the potential impact of AvSTAR derived 
technology on the ATM system in the years 2010 and 2025. The task will involve 
coordination with NASA personnel and up to two other NASA support 
contractors. 

In the context of this scope of work, NASA requested that Boeing ATM focus the 
AvSTAR assessment on providing answers to the following three questions: 

1 . Goals : Are the stated AvSTAR goals complete and correct with respect to the 
needs of the U.S. National Airspace System (NAS) for the stated timeframe? In 
other words, do the goals address the real needs of the NAS, and are there any 
additional goals that should be included? 

2. Feasibility : Are the goals technically feasible; i.e., could they be achieved in the 
given timeframe if adequate resources were allocated? 

3. Plan : Does the AvSTAR project plan provide the technical direction, resources, 
and coordination necessary to achieve the goals in the given time? 

The approach of the Boeing team in gathering information was to attend the AvSTAR 
government/industry workshops held in September 2000 and March 2001, hold 
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teleconferences with AvSTAR personnel (weekly or biweekly), and analyze AvSTAR 
project presentation materials. The Boeing team also conducted regular internal meetings 
in order to discuss findings and recommendations. Finally, an air traffic demand-capacity 
study using the Boeing Current Market Outlook (CMO) was performed to provide an 
initial assessment of the feasibility of the NASA goals. This work was performed jointly 
by the Mathematics and Computing Division of Phantom Works (Boeing’s research 
organization) and Boeing’s CMO organization. 

This report is organized around the three questions that were addressed, with the goals 
stated in section 2, and analysis and answers presented in sections 3-5. Section 6 presents 
summary conclusions and recommendations for the AvSTAR project office, and 
appendices A and B present further details supporting the definition of demand, 
throughput, delay, and the CMO-based analysis perfonned. 


2. Goals 

The objectives and goals of the AvSTAR project, as defined in the presentation by Dallas 
Denery at the March 2001 workshop [1], are: 

Objectives : 

Provide the research and development by 2007 necessary to: 

1 . Complete the development of technology for tomorrow (Free Flight). 

2. Provide the foundations for setting the future direction (Beyond Free Flight). 

Goals : 

Tomorrow’s Air Transportation System (2010) 

1 . 22-percent increase in throughput at high-density airports 

2. 25-percent reduction in today’s missed/cancelled flights due to traffic problems 

3. 25-percent reduction in average delay 

The Future Air Transportation System (2025) 

1 . 64-percent increase in throughput at high-density airports 

2. 50-percent reduction in missed/cancelled flights 

3. 50-percent reduction in average delay 

Finally, in this project, air transportation is viewed as one component of a fully integrated 
multi-modal transportation system. 


3. Are the Goals Complete and Correct? 

The throughput goals are lower than the increase in demand predicted by the Boeing 
Current Market Outlook 2000 [2]. The NASA figures of 22-percent and 64-percent 
increases for 10 and 25 years, respectively, are based on a 2-percent average annual 
increase in air traffic. The Boeing CMO for the next 10 years currently predicts a 2.8- 
percent annual increase in revenue passenger kilometers (RPKs) for the North American 
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region, leading to a total increase in departures of 77 percent by 2020. This includes 
Canada, but over 90 percent of the flights are U.S. based. Also, this does not include 
international flights that either originate or terminate in the U.S., which have been 
growing at a somewhat higher rate over the last decade. 

Recommendation 1. Revise the 2-percent growth assumption in the goals statement 
related to projected increases in future traffic demand. See Recommendation 5 for 
additional details. 

The AvSTAR project schedule for the completion of research to support these goals is 
about right: finish the research for the 2010 goals by 2007 and finish research for the 
2025 goals by 2015. Based on the length of time between the end of each research phase 
and the corresponding start of implementation, the authors recommend achieving a 
technology readiness level (TRL) of at least 6 by the end of the 2007 phase (for 2010 
goals) and a TRL of at least 4 at the end of the 2015 phase (for 2025 goals). (Note: The 
March 2001 plan provided only for advanced component technologies to go to TRL 1 or 
2. The proposed AvSTAR augmentation plan is structured to achieve TRL 4 by 
2007.) 

Recommendation 2. The schedule is aligned with the growth assumption. However, 
a clearer distinction needs to be made between the NASA research goals and the 
NAS implementation goals. 

The goal statements would be better understood if they were accompanied by definitions 
of the basic concepts and metrics used (demand, throughput, and delay), and their 
relationships. Appendix A discusses how these terms could be defined to avoid 
ambiguous interpretation and potential difficulty in demonstrating that the goals will be 
met. 

Recommendation 3. Clearly define the metrics used for goals. 

One may ask if an unqualified increase in average system throughput will, in fact, result 
in a large proportion of passengers being able to fly when and where they want. Boeing 
performed a study, described in more detail in appendix B, which shows how future air 
carrier flight schedules are likely to become distorted if demand continues to increase but 
capacity does not. This study shows one estimate of the effect on “schedule quality” (a 
simple measure of whether or not passengers can fly when they want to) as a function of 
varying future capacity growth at today’s high-density airports. As demand increases 
over the next few years, with little or no corresponding increase in capacity, a greater 
portion of the passengers will be forced to depart or arrive at undesirable times of day. 
Undesirable times are determined from a standard Boeing plot of passenger preferences 
(shown in Figure B.3-1 and discussed in more detail in appendix B). 

The analysis indicates that if NASA’s throughput goals were met, there would be no 
significant change in this schedule quality value at Chicago, Atlanta, or anywhere across 
the whole NAS. Relative to this schedule quality metric, the AvSTAR goals seem to be 
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aligned with the currently predicted growth in demand. As discussed in appendix B, this 
simple schedule quality metric does not capture the complexity of passenger time-of-day 
travel preferences and constraints (particularly for the business traveler), but is only a 
first-order indication of throughput goal adequacy. 

Recommendation 4. Link the throughput goals to metrics of passenger (and other 
user) preferences and include studies to quantify performance against such metrics. 

The NASA throughput goals are based on industry predictions of future growth in 
demand for air traffic. As pointed out at the beginning of this section, the Boeing CMO 
differs in growth-rate prediction from the assumption made for establishing the current 
AvSTAR throughput goals. One thing is certain: all traffic growth predictions have been 
incorrect in one aspect or another. To hedge against the difficulty in forecasting traffic 
growth, the AvSTAR program should consider a range of forecasts, generated by making 
several different assumptions about the primary factors that influence growth. These 
factors include economic growth in different regions of the NAS, airline business and 
operational strategies, airplane size and perfonnance mix, policy decisions that affect 
other modes of transportation, and so on. These future scenarios can result in 
significantly different future traffic patterns, for which NASA should attempt to have a 
robust set of solutions. 

Recommendation 5. Look at a range of potential future scenarios, and tailor the 
goals and project plan to accommodate them. 

Two factors that should be considered in the goal-setting process are other ( i. e. , non- 
NASA) projected improvements to the NAS and technical risk, ft might be possible to 
scale back the AvSTAR goals somewhat based on predicted capacity increases from 
initiatives outside NASA’s scope. Conversely, it might be necessary to make the goals 
more ambitious to compensate for the possibility that some concepts or technologies have 
a significant likelihood of not being implemented. 

On the favorable side of this question, consider, for example, potential new runway 
construction occurring in the given timeframe. This is an example of a system capacity 
factor outside NASA’s project scope. Figure 1 shows runways as one of several places on 
a particular flight where bottlenecks might occur. The FAA Operational Evolution Plan 
(OEP) [3] states that new runways will be constructed at 15 airports between now and 
2010 (ATL, IAH, DFW, PHX, 1AD, STL, DTW, CVG, MSP, MIA, SEA, MCO, CLT, 
DEN, and BOS). The new runway at Boston will be used only in special circumstances, 
so it is not considered a basic capacity increase factor. The new runways at the 14 
remaining airports are projected to produce an average capacity increase of 28 percent 
per airport in visual meteorological conditions (VMC) and 37 percent per airport in 
instrument meteorological conditions (IMC). The FAA’s OPSNET database shows that 
these 15 airports handle about 30 percent of the operations recorded per year at the top 99 
airports in the NAS. This runway investment plan should be analyzed to produce an 
estimate of the capacity increase benefit across the entire system and thus on the NASA 
ATM system throughput goals. 
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Figure 1. Potential System Bottlenecks. 


Runways, however, are not the only NAS resource that can pose a constraint to traffic 
flow, and Figure 1 illustrates numerous other operational regions that also must be 
considered. Of those, NASA’s primary research and development charter is in the 
management of surface. Traffic Management Advisor (TMA), and en-route areas, all of 
which have been shown to contribute to some portion of the predicted capacity shortfall. 
If a sufficient number of runways were added to the system to handle the growth, would 
this be sufficient to ensure overall system throughput with low levels of delay? The 
concern is that it would not, that, in fact, the TMA and en-route airspace will soon 
become the primary bottleneck in many areas of the NAS, or that at the largest airports, 
surface movement capacity would limit the actual throughput increase achievable with 
new runways. 

Therefore, the translation of the overall AvSTAR throughput, delay, and cancellation 
goals into performance objectives for the various system operating regions is not 
straightforward, and an early project activity needs to provide a clear understanding of 
what improvements are needed and where they should be implemented. It may be 
possible to scale back on the performance goals for one operating region, while the need 
for another might be even greater than what is now predicted. The tools used in the 
Boeing demand-capacity study described in appendix B, as well as other national analysis 
tools that consider only airport capacity, are inadequate to provide insight into the 
required regional performance goals, as discussed in the Boeing paper "ATM Operational 
Concepts and Technical Performance Requirements" [4]. The AvSTAR project should 
have such total system performance tools developed in the first phase of work, and apply 
them to establishing specific performance goals for the operating regions for which the 
project has been given research charter. This will require an initial focus on the 
development and integration of fast-time modeling and assessment tools that can address 
total system performance. 
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Recommendation 6. Performance goals for particular operating regions need to be 
based on a total system performance analysis and an allocation of performance 
requirements to each operating region. 

The high-level system throughput goal does not address the various capacity levels of the 
current NAS under different operating conditions, which contribute most significantly to 
the delay and cancellation performance goals. Allocation of throughput and/or delay 
goals to major weather categories such as IMC, VMC, and convective weather may be 
necessary to ensure that various technology solutions deliver a robust overall system 
performance. Appendix B illustrates that some of the predicted traffic growth may be 
accommodated by “filling in” the current off-peak times in the airline schedule. 
Considering that the schedule is based largely on the good VMC performance at each 
airport, it is clear that a schedule that is saturated at the VMC capacity level throughout 
the day has no room to recover from periods of poor weather. To achieve the delay and 
cancellation reduction goals, it is imperative that the difference in VMC and IMC 
performance levels be kept to a reasonable minimum, limited primarily by considerations 
of single-flight safety concerns. 

Recommendation 7. Define performance goals for particular operating conditions, 
such as IMC Category I, II, and III, and convective weather. These goals should be 
as close as possible to the VMC goals, and they should be derived from the delay 
and cancellation goals. 

Technical implementation risk is another concern that the AvSTAR project plan should 
account for in setting goals, as well as in the overall project execution. One possible 
approach to accounting for implementation risk would be to set the system performance 
goals somewhat higher than what is predicted to be needed. This way, if certain elements 
of the solution set are not successfully implemented, the end result has a higher 
probability of meeting the system need. Care should be taken not to set the mark so high 
that a solution becomes infeasible, and to make adjustment during execution of the 
project to continue tracking toward the goals. 

Safety is mentioned in several AvSTAR presentations, but in the authors’ opinion it is not 
given nearly enough emphasis. Capacity and safety are inextricably intertwined, and an 
effort to increase one may, if care is not taken, result in decrement to the other. Given the 
safety-critical nature of the air transportation system, and the agreed high level of safety 
demanded by the traveling public, it is imperative that all research and development for 
ATM be performed with a clear safety goal in mind. Safety requirements are often the 
driver for fundamental system design decisions, and they can strongly influence the 
system functional and physical architecture. Although the consideration of safety 
objectives may appear initially to slow the process and constrain the set of potential 
solutions, their inclusion from the first phases of research is the best approach to mitigate 
technical risk. The ambitious goals set for the AvSTAR project will require system 
designs radically different from today’s, and the process of commissioning and/or 
certifying the new system will be challenging. The successful implementation will hinge 
upon a demonstrated safety performance, above all other considerations. Therefore, it is 
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recommended that clear statements about safety be added to the goals, and that 
appropriate safety modeling and analysis methodologies be integrated into the plan from 
the start. 

Recommendation 8. Safety goals and metrics should be included explicitly in the 
goals statement and the project plan. 

Another potential system goal to be considered is access. Access to airspace may involve 
the allocation of airspace use among user categories instrument flight rules (IFR) and 
visual flight rules (VFR) and associated equipage requirements. Access is viewed to be 
especially important to general aviation, military, and other user groups. 

Recommendation 9. Include access goals for all major airspace users. 

Environmental issues must be considered in terms of airport noise and emission levels to 
be supported in future operations. With traffic growth, these considerations may be 
increasingly important for technology assessment. 

Recommendation 10. Include environmental issues in the goals statement. 

Finally, cost is a performance metric that is key to acceptance of new systems and 
eventual investment by all stakeholders. There is little doubt that given the current NAS 
performance levels and the predicted growth, the solution to the 2025 perfonnance 
problem will involve a major investment across the system. This could prove to be the 
most significant risk to success, particularly if not explicitly accounted for in the goal. In 
general, the industry is willing to make significant investments, provided that a clear case 
can be made that correspondingly significant benefits will be derived. Thus, an affordable 
solution can be one that carries high cost, if the benefits are even higher. 

Recommendation 11. Include affordability in the goals statement. 


4. Are the Goals Technically Feasible? 

Boeing ATM’s view of the technical challenges of air traffic growth, as expressed in the 
June 2001 presentation “Air Traffic Management” [5], is that an engineering solution to 
address this challenge is technically feasible. The Boeing ATM plan, when combined 
with the FAA OEP [3], is predicted to accommodate 15 to 17 years of traffic growth in 
the NAS. Furthennore, when combined with new runway construction and advanced 
airport concepts, 20 to 25 years of traffic growth can be accommodated. The most 
significant point of difference between the NASA and Boeing plans is the 
implementation schedule, which in the Boeing plan targets the initial operating capability 
of the last system phase in 2008. The NASA target is no earlier than 2015. The Boeing 
plan explicitly states that additional runway capacity will be required to satisfy the 
demand, but the AvSTAR project material is silent on this issue. 
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AvSTAR and the Boeing plan agree on the nature of the change that will be required in 
ATM. Both plans discuss the need for a radically new paradigm in ATM, one that 
includes significantly increased levels of automation across the system. It is Boeing’s 
view that NASA’s research and development charter in U.S. ATM is unique in its long- 
term objectives, and that a fundamental research focus on this challenging problem 
should be a high priority for the U.S. to ensure the viability of its transportation system. 
U.S. industry has neither the resources nor the required expertise and facilities to perform 
this research, but stands ready to implement new concepts and technologies as they 
become mature and can be shown to have significant benefits potential. 

In light of the very limited progress made over the last two decades in implementing new 
technology for ATM, it is clear that significant risks are present that could prevent the 
system from satisfying the predicted demand. The primary risks, in the authors’ opinion, 
include the following: 

• Air traffic controller resistance to increased automation and significantly changed 
working methods 

• Local residential resistance to airport capital investment for increased throughput 

• General aviation pilot resistance to more expensive equipment required to operate 
in high-density airspace 

• Airline resistance to invest in new technology and change operating methods, 
particularly in light of opportunities for existing strong players to profit (without 
any investment) in a constrained market 

• (Related to the last item) Mixed equipage: new operations designed to increase 
capacity may not be fully exploited when some of the participants do not have the 
requisite equipment 

• Intermodal transportation improvements may not keep pace with traffic growth, 
thus limiting the public’s access to the air transportation system 

• Inability to solve the wake vortex problem on final approach, thus making the 
system even more sensitive to low-visibility conditions than it is today 

• Certification of new ATM systems that include complex software and high levels 
of integration across a large number of agents 

• Resistance from current communications, navigation, and surveillance 
(CNS)/ATM equipment suppliers to potential major changes in system 
infrastructure that could change that marketplace 

This list of risk factors is by no means comprehensive, but all have been active forces in 
the NAS modernization process over the last decade. To increase the probability of 
success of the AvSTAR project, NASA must design and implement a risk management 
plan with the program, clearly identifying all important risk elements. NASA should 
address the factors it can include in its research program, and also alert the FAA and the 
Department of Transportation to critical factors that are outside of NASA ’s control. It is 
in the best interest of NASA, as well as the NAS, to identify and openly discuss all 
important risk factors. Otherwise, the AvSTAR project may promise results that are 
unlikely to be achieved because of external factors. 
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Recommendation 12. Establish a risk management plan for factors both internal 
and external to NASA. 


5. Is the NASA Plan Appropriate for Achieving the Goals in the 
Allotted Time? 

The Boeing team based its efforts at answering this question on AvSTAR project 
presentation material [1] and on numerous conversations with NASA staff actively 
involved in planning the project. Given the preliminary and conceptual nature of the more 
visionary elements of the project, this evaluation had to be performed at a very high level, 
and a significant amount of expert judgment had to be applied to predicting the likely 
outcome of each project element. Thus, the results have to be considered as extremely 
preliminary, and unavoidably biased by the experience and opinions of the individuals on 
the team. An attempt is made to compensate for this by providing explanatory comments 
for the assessment of each project element, and it is the hope of the authors that the use of 
this material be limited to making additions and/or improvements to the definition and 
execution of the work undertaken during the project. 

General Remarks 

The AvSTAR project plan assessed in this report, as defined in the March 2001 workshop 
presentations, is grouped into four major categories: System-Level Concept Development 
and Evaluation (SLCDE), Tomorrow’s Component Technologies, Advanced Component 
Technologies, and Virtual Airspace System Technology (VAST). 

The Tomorrow’s Component Technologies area is intended to complete research and 
development on technologies that were started under the Advanced Air Transportation 
Technologies (AATT) program, in support of the FAA’s midterm perfonnance needs. As 
such, these technologies were originally constrained to augment the current ATM 
paradigm by providing incremental automation capability to the current working 
positions, assuming little or no change in CNS infrastructure. The exceptions are some 
elements of the Distributed Air-Ground Traffic Management (DAG-TM) concept 
involving airborne self-spacing, and some data-link capability assumptions. 

The SLCDE component includes two key elements: the development of alternate 
operational concepts and the ability to perform benefits trade studies on these concepts. 
The plan to develop alternate concepts is the logical first phase of such a program, but 
few details are provided on how this will be undertaken. The automated airspace concept 
presented at the workshops is a good first example, and other concepts are presumed to 
follow. It is not clear to what extent the concept generation activity will be driven by the 
system performance goals stated for the program, or a less structured set of potential 
concept variations conceived for the ATM field. 
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Recommendation 13. The concept development activity should be firmly guided by 
the system performance goals, particularly throughput and safety. 

The evaluation component of the SLCDE effort is vital to the process of discriminating 
among the various concepts that may be generated. The VAST effort is intended to 
support this evaluation with an appropriate toolset. In the beginning, the bulk of the 
VAST effort should support the SLCDE effort, via simulations designed to supply 
models or parameters needed for the operational concept analysis and broad concept trade 
studies. This cannot be achieved efficiently with human-in-the-loop (HIL) methodology, 
but will require a fast-time simulation capability. Only later, for mature, highly promising 
concepts and technologies, should the VAST project venture into the ambitious enterprise 
of connecting vast arrays of labs, computing facilities, operational centers, etc. into a 
large-scale distributed simulation capability. 

Recommendation 14. The VAST component should emphasize the development of a 
fast-time operational concept performance-analysis capability. 

The evaluation tools of the SLCDE effort should include the capability to perform trade- 
offs for safety as well as capacity. In order to estimate the overall system safety, one has 
to consider the interactions of the various subsystems, technologies, and procedures with 
each other. The fast-time models should include the ability to simulate and quantify the 
effects of rare-nonnal and abnonnal behavior. The VAST real-time high-fidelity HIL 
modeling can be used to investigate human factors issues and provide parameters and 
behaviors for the fast-time overall system models. 

Recommendation 15. The VAST component should include an ATM safety-analysis 
capability, including representation of human performance elements. 

The government/industry workshops have included all the stakeholders and have 
contributed to an understanding of other stakeholders' points of view. It is clear that 
NASA is addressing feedback from the other industry stakeholders. In the March 2001 
workshop, for example, the NASA presentations contained responses to the comments 
and questions from the previous workshop. 

Detailed Program Task Assessment 

In this section, specific tasks of the AvSTAR program are evaluated based on their 
contribution to the overall program goals. The tasks are defined in the March 2001 
workshop presentations, and are grouped into four major categories: SLCDE, 

Tomorrow’s Component Technologies, Advanced Component Technologies, and VAST. 

Each task element is assessed for its potential contribution (assuming the end product is 
implemented) to the capacity and safety of the NAS. Capacity and safety assessments are 
based on the apparent emphasis in these areas by the material provided in the NASA 
AvSTAR plan, and the authors' understanding of the likely content of the work statement. 
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The cost of the end product of each task element is evaluated, based on a purely relative 
and qualitative best-judgment estimate of the investment cost to the air traffic service 
provider (ATSP) and airspace users, as well as the potential reductions in operations and 
maintenance cost. These cost evaluations are based solely on the authors’ assessment of 
their potential effect on the NAS, because the AvSTAR program material does not yet 
address these perfonnance metrics. Note that a High rating for cost is not necessarily a 
detriment, because significant performance increases can be expected to require an 
equally significant investment. 

A rating of N/A was used numerous times, to indicate that a task either had no relevance 
to the metric, or that insufficient infonnation was present to make an evaluation. 

Finally, whenever possible, additional explanation or clarification is provided, in the form 
of remarks. 

System-Level Concept Development and Evaluation 
Key Element 1 . 

Develop integrated methods for system-level analysis: investigate, develop, validate, and 
document methods for model-based system-level analysis; human system modeling; and 
analytic methods for hybrid systems, including human and team modeling. 

Capacity: High 

Safety: High 

Operational and Maintenance Cost Reduction: Possible, if explicitly targeted. 

Investment Cost to ATSP and Users: N/A 
Remarks: 

This capability is intended to reduce the CNS/ATM industry's reliance HIL 
experimentation for early concept development, which tends to take a long time from 
initial research to implementation in the field. This toolset has the potential to include 
safety assessment in early concept studies, including the role of the human operators, 
which can help ensure that automation technologies can be certified when ready for 
deployment, even for reduced traffic-separation standards. It will enable efficient 
development of a new air/ground integrated ATM paradigm. There is also a potential to 
reduce ATSP operations cost if targeted by the research. 

Recommendation 16. Increase the emphasis on, and therefore the budget for, the 
SLCDE portion of the program. 

Key Element 2. 

Gather, sustain, and analyze operational concepts: identify and define concept evaluation 
criteria and conduct concept feasibility trade-off evaluations. 
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Capacity: Medium 
Safety: Medium 

Operational and Maintenance Cost Reduction: Possible, if explicitly targeted. 

Investment Cost to ATSP and Users: N/A 
Remarks: 

This needs to be performance driven, rather than a random collection exercise across the 
entire spectrum of potential concept parameters. The key is the ability to evaluate and 
select the concepts with the highest potential against the most important system 
performance metrics. Thus structured, this would get high marks for capacity and safety. 

Recommendation 17. Focus the concept development process on achieving the most 
important system performance objectives. 

Key Element 3. 

Validate and recommend candidate concepts for the NAS: infrastructure characterization; 
traffic scenario generation; HIL simulation; batch simulation; translation to nationwide 
performance; and benefits and impact analysis. This will make use of the VAST 
capability. 

Capacity: High 

Safety: High 

Operational and Maintenance Cost Reduction: Possible, if explicitly targeted. 

Investment Cost to ATSP and Users: High 
Remarks: 

As in Key Element 1, safety analysis should be integrated into this element. For example, 
HIL simulation can be used to derive performance and error-rate parameters for the batch 
simulations. The overall simulations should consider the safety implications of various 
interfaces (e.g., sector-to-sector, human-machine, pilot-controller) and how they are 
affected when new operational procedures or technologies are introduced. 

Key Element 4. 

Transfer analysis products to the FAA: identify and document AvSTAR transition 
approach. 

Capacity: N/A 

Safety: N/A 
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Operational and Maintenance Cost Reduction: N/A 
Investment Cost to ATSP and Users: N/A 

Remarks: The metrics do not really apply in this case. However, risk assessment of 
implementing any of the technologies is strongly encouraged, and the plan can be 
structured to reduce this risk. 

Tomorrow’s Component Technologies 
Key Element 1 . 

Surface congestion alleviation: surface movement decision support tools (DSTs), 
advisories; displays; and data link. 

Capacity: Low 

Safety: Medium 

Operational and Maintenance Cost Reduction: Low 
Investment Cost to ATSP and Users: Medium 
Remarks: 

Safety benefit depends upon successful inclusion of runway incursion-prevention 
functionality. 

Key Element 2. 

Runway productivity: Aircraft Vortex Spacing System (AVOSS) and Airborne 
Infonnation for Lateral Spacing (AILS); AILS submitted as “over-guideline” budget 
request 

Capacity: High 
Safety: High 

Operational and Maintenance Cost Reduction: Low 
Investment Cost to ATSP and Users: High 
Remarks: 

AILS and AVOSS have the potential to increase capacity and safety if they achieve user 
acceptance. 

Key Element 3. 

Arrival/departure decision support tools: Center/TRACON Automation System (CTAS), 
active Final Approach Spacing Tool (aFAST), Expedite Departure Path (EDP); DAG- 
TM, dynamic routing and spacing; interdependent arrival/surface advisories; airborne 
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self-spacing; and environmentally compatible operations; dynamic routing and 
environmental compatibility will be taken to TRL 4 only 

Capacity: Medium 

Safety: N/A 

Operational and Maintenance Cost Reduction: Low 
Investment Cost to ATSP and Users: Medium 
Remarks: 

This element includes a large collection of AATT technologies, most of which were 
developed assuming the current control paradigm. Although these tools will not provide 
significant increases in capacity when installed within the current control paradigm, they 
are an essential step in moving toward a more automated airspace that has a potential for 
capacity gains. DAG-TM (in particular, concept #11) has a potential for providing 
capacity gains via reduction of separation, and individually would be rated High in the 
Capacity category. The safety benefit cannot be assessed because no analysis has been 
provided by the AATT project. 

Key Element 4. 

Integrated airspace decision support tools: en-route descent advisor (EDA); direct-to 
tools; regional flow metering; constrained airspace tool; air traffic control/Airline 
Operations Center (ATC/AOC)/flight-deck integration. The spacing tool is dropped, and 
regional flow metering work is accelerated (will make miles-in-trail obsolete). 

Capacity: Medium 

Safety: N/A 

Operational and Maintenance Cost Reduction: Low 
Investment Cost to ATSP and Users: Medium 
Remarks: 

See the previous element. En-route airspace in the U.S. is not a significant capacity 
constraint, except in cases of significant convective weather activity. Although the most 
severe delays occur during these periods of adverse weather, such periods are relatively 
infrequent. As a result, even the most effective combination of strategies and/or tools 
would result in a medium capacity improvement at best. The authors think a proper 
combination of the constrained airspace tool (which would help in convective weather 
en-route situations) and the ATC/AOC/flight-deck integration (which would help reduce 
sector workload) has a potential for providing a Medium capacity improvement. 
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Key Element 5. 

National traffic flow management: Traffic Flow Automation System (TFAS) and Future 
ATM Concepts Evaluation Tool (FACET). TFAS is consolidated under AATT; this 
activity is integrated with related FAA activities through the Inter-Agency Integrated 
Product Team (IAIPT) (Mitre Center for Advanced Aviation System Development 
(CAASD) and Volpe). 

Capacity: Fow 

Safety: N/A 

Operational and Maintenance Cost Reduction: Fow 
Investment Cost to ATSP and Users: Fow 
Remarks: 

The element should contribute to the delay and cancellation reduction goals, particularly 
related to weather-induced capacity constraints. 

Key Element 6. 

Small airport utilization: the cost of this new activity is currently $3.5 million. 

Capacity: More access for general aviation (GA). 

Safety: Medium improvement for GA. 

Operational and Maintenance Cost Reduction: Fow 
Investment Cost to ATSP and Users: Medium 
Remarks: 

More access for the GA user segment may pose an increase in load on the overall ATM 
system beyond that predicted by growth in commercially scheduled operations. Thus, 
traffic growth predictions must be accounted for in this segment of the user class. 

Key Element 7. 

Runway-independent aircraft (A/C) operations: simultaneous noninterfering (SNI) 
operations; vertical/short takeoffivertical landing (V/STOVF) operations; low noise; 
missed approaches. Submitted as an “over-guideline” budget request, this element has 
been split into SNI and adverse weather/low noise operations; it will be integrated with 
FAA plans in GA and vertical flight technology. 

Capacity: Medium 

Safety: Fow 
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Operational and Maintenance Cost Reduction: Low 
Investment Cost to ATSP and Users: High 
Remarks: 

Runway-independent operations and simultaneous operations may reduce traditional 
safety margins and, therefore, need compensating technologies and/or procedures. Low- 
noise approaches achieved via approach-path modifications involve increased complexity 
during a time of high workload. V/STOVL operations introduce a dissimilar element into 
a heretofore fairly homogenous environment. Therefore, the components of this key 
element need a careful and extensive safety analysis. The overall CNS/ATM 
infrastructure for V/STOVL operations needs significant improvement, including 
airspace and procedure design criteria, which will be costly. 

Key Element 8. 

ATM/traffic flow management (TFM) weather integration: focal point for all weather 
requirements for decision support tools (DST). 

Capacity: Medium 

Safety: Medium 

Operational and Maintenance Cost Reduction: Low 

Investment Cost to ATSP and Users: Medium 

Advanced Component Technologies 
Key Element 1 . 

ATM automation technologies: real-time system optimization; four-dimensional (4-D) 
optimized ATC in terminal area; increased traffic density/reduced sector boundary 
constraints; automated A/C separation using DAG conflict detection/resolution; high- 
level automated traffic flow control. DAG concepts and aircraft self-separation are 
included. 

Capacity: High 

Safety: Low 

Operational and Maintenance Cost Reduction: High potential to reduce ATSP operations 
cost if targeted by the research. 

Investment Cost to ATSP and Users: High 

Remarks: This element needs to focus on good design principles to keep complexity in 
check, as well as a thorough safety analysis. The plan appears to have insufficient safety- 
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related content to address the significant safety and certification issues. Overall, the 
concept is very promising from system capacity and affordability points of view. 

Key Element 2. 

Controller/pilot/system human factors in a highly automated airspace: roles and 
responsibilities; supervisory control and goal setting; human interface requirements; 
advanced en-route and tenninal automation; advanced surface ATM automation; 
interfaces for automation-assisted flight planning and communication. 

Capacity: High 

Safety: High 

Operational and Maintenance Cost Reduction: Medium 
Investment Cost to ATSP and Users: High 
Remarks: 

Include consideration of human factors issues, especially for system safety aspects. The 
high marks are for including human factors in the project from the concept stage, because 
it is expected that this will force a focus on safety in nonnormal operating modes from the 
start. 

Key Element 3. 

Infonnation infrastructure technologies: CNS technologies; information system 
architecture; requirements for ATM information flow; development and assessment of 
candidate architectures; identification of technology “gaps”. 

Capacity: High 

Safety: Low; not enough emphasis 

Operational and Maintenance Cost Reduction: Potential, if NAS architecture can be 
simplified and a significant number of old infrastructure elements decommissioned. 

Investment Cost to ATSP and Users: High 

Remarks: 

A potential risk element here is that it will require a radical change in the FAA 
architecture, with associated certification challenges. 


Key Element 4. 

Improved utilization of airports/limited airspace: meta-airport operations; closely spaced 
operations; dynamically configurable runways/taxiways; zero-visibility surface 
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movement; virtual tower; airport robotics. This element would attempt to challenge the 
one-aircraft-per-runway rule. 

Capacity: High+ 

Safety: Medium 

Operational and Maintenance Cost Reduction: N/A 
Investment Cost to ATSP and Users: High 
Remarks: 

This element needs an integrated safety analysis because there are so many interacting 
components and aircraft density is higher around airports. This has the highest potential 
for capacity increase of all tasks, assuming the primary bottleneck is the runway. 

Virtual Airspace Simulation Technology (VAST) 

Key Element 1 . 

Create a virtual research environment: remote accessibility to labs, and simulation 
environments. 

Capacity: N/A 

Safety: N/A 

Operational and Maintenance Cost Reduction: N/A 
Investment Cost to ATSP and Users: N/A 
Remarks: 

Some distributed simulation capability may be useful for AvSTAR, but in the later stages 
of the project to support the technology transfer process. This element will be very costly 
to establish, and very costly and time-consuming to perform experiments and 
demonstrations, so it is not appropriate for concept exploration and early stages of 
research. 

Key Element 2. 

Link existing and future simulation concepts: software emulations and HIL simulation 
facilities. 

Capacity: N/A 

Safety: N/A 

Operational and Maintenance Cost Reduction: N/A 
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Investment Cost to ATSP and Users: N/A 


Remarks: 

The HIL simulations can provide valuable data to fast-time simulations, and thereby 
contribute to safety if the fast-time simulations include rare-normal and abnormal 
behaviors. 

Key Element 3. 

Establish network architecture: link multiple facilities/capabilities in rcconfigurable 
groupings. Initial work will establish the requirements for this architecture. 

Capacity: N/A 

Safety: N/A 

Operational and Maintenance Cost Reduction: N/A 
Investment Cost to ATSP and Users: N/A 


6. Summary of Assessment and Recommendations 

The authors commend the AvSTAR project planning team at NASA for the following 
important aspects of the current plan: 

• a statement of the ATM system throughput needs through 2025, in the form of 
quantifiable goals for this research project; 

• a significant emphasis on quantitative assessment of proposed new operational 
concepts, and an ambitious plan to establish a new, fast-time capability to perform 
such assessment early in the project; this will enable NASA to select the most 
promising new ATM concepts based on system performance goals, thus focusing 
research clearly on NAS perfonnance; 

• presentation of a possible new ATM paradigm that has the potential to 
significantly increase the system capacity and affordability, along with a plan to 
generate numerous other new concepts; and 

• explicit involvement of all government and industry stakeholders in the project 
definition phase, by hosting two major workshops to present NASA’s ideas and 
solicit stakeholder input. 

As discussed in more detail in earlier sections of the report, the authors recommend the 
following revisions to the current AvSTAR plan: 

Recommendation 1. Revise the 2-percent growth assumption in the goals statement 
related to projected increases in future traffic demand. 

Recommendation 2. The schedule is aligned with the growth assumption. However, 
a clearer distinction needs to be made between the NASA research goals and the 
NAS implementation goals. 
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Recommendation 3. Clearly define the metrics used for goals. 

Recommendation 4. Link the throughput goals to metrics of passenger (and other 
user) preferences and include studies to quantify performance against such metrics. 

Recommendation 5. Look at a range of potential future scenarios, and tailor the 
goals and project plan to accommodate them. 

Recommendation 6. Performance goals for particular operating regions need to be 
based on a total system performance analysis and an allocation of performance 
requirements to each operating region. 

Recommendation 7. Define performance goals for particular operating conditions, 
such as IMC Category I, II, and III, and convective weather. These goals should be 
as close as possible to the VMC goals, and they should be derived from the delay 
and cancellation goals. 

Recommendation 8. Safety goals and metrics should be included explicitly in the 
goals statement and the project plan. 

Recommendation 9. Include access goals for all major airspace users. 

Recommendation 10. Include environmental issues in the goals statement. 

Recommendation 11. Include affordability in the goals statement. 

Recommendation 12. Establish a risk management plan for factors both internal 
and external to NASA. 

Recommendation 13. The concept development activity should be firmly guided by 
the system performance goals, particularly throughput and safety. 

Recommendation 14. The VAST component should emphasize the development of a 
fast-time operational concept performance-analysis capability. 

Recommendation 15. The VAST component should include an ATM safety-analysis 
capability, including representation of human performance elements. 

Recommendation 16. Increase the emphasis on, and therefore the budget for, the 
SLCDE portion of the program. 

Recommendation 17. Focus the concept development process on achieving the most 
important system performance objectives. 
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Appendix A. Definitions of Demand, Capacity, Throughput, and Delay 

The terms used to describe how much traffic the air transportation system can handle are 
often used interchangeably, and with different interpretations by different individuals. To 
avoid miscommunication, the following definitions used in this report are recommended 
to NASA: 

Demand is the number of aircraft per unit of time attempting to pass through an airport or 
some specified volume of airspace. This quantity, for airports, is estimated in the Current 
Market Outlook (CMO) based on assumptions about economic growth and airline 
business and operational strategies. 

Throughput is the number of aircraft per unit of time actually passing through an airport 
or a given volume of airspace. The most commonly used time units are quarter hour, 
hour, and day. Throughput is easily recorded by simply counting the number of 
operations at a location in a given time period, and clearly depends on the presence of 
traffic demand. 

Delay can be defined as the difference between scheduled and actual times at some 
specified point (runway threshold or gate, for example). Delay can be measured relatively 
easily by recording times at defined positions. 

The primary practical difficulty with delay measurement is that the airlines tend to 
manage their schedule (e.g., with larger block times) to reduce the operational impact of 
likely delays, in fact, reported average delays, as measured against the airline schedule, 
have not changed much in a decade [6]. The consumer-oriented on-time arrival reporting 
system has almost certainly contributed to this effect. Consequently, there is a significant 
buffer built into the flight schedule, which represents a significant cost in terms of 
equipment and manpower utilization and is difficult to account for using a traditional 
operational delay analysis. 

On the other hand, as the U.S. National Airspace System (NAS) reaches saturation, 
events such as weather perturbations can cause large, widespread delays that attract a lot 
of media attention. Therefore, it might be beneficial to focus some attention on reducing 
the sensitivity of the system to such perturbations, so that the large variations in delay do 
not occur, or at least occur less often. 

Capacity can be defined as the maximum theoretical throughput of the whole system, or 
of subsets of the system such as airports or air traffic control sectors. System capacity is a 
quantity of interest because it represents a theoretical limit on what might be 
operationally achievable. 

The actual computation of system capacity requires a sophisticated analysis of the air 
traffic network, including an assessment of the capacity of the human operators to handle 
traffic demand under some ideal and nonideal conditions. There would be several 
different capacity results, depending on the assumptions made about components of the 
system, external factors such as weather, and demand levels. 
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Nevertheless, it is possible to approximate the relationship among demand, capacity, and 
delay by making some simplifying assumptions. In particular, assume the system (NAS, 
tenninal area, or airport, for example) can be modeled as an M/M/1 queue (single-server 
system with infinite storage capacity and exponentially distributed inter-arrival and 
service times) [7]. This is a major simplification, but it illustrates some of the phenomena 
that would be present in a more complete analysis. 

In this model, let X be the arrival rate of aircraft (demand) and p be the service rate. Let 
p = X/p, the traffic intensity. Then a standard result from queuing theory [7] says that the 
average waiting time in the system is given by 


W = 


P 

A(l-p) 


1 


u - A 


Note that l im W = — , which says that, when demand is very low, the average waiting 

A ^° u 

time is the average service time, which is what would be expected. Define the delay D in 
this simple model to be the excess of the waiting time over the average service time; that 
is, 


u u - A 

Figure A-l shows a plot of delay D vs. demand X in this simple queuing model. Clearly, 
delay is very sensitive to demand in the region where demand rate is close to the average 
service rate. This illustrates the well-known phenomenon of rapidly escalating delays in a 
system that experiences even a small increase in demand while operating close to 
capacity. 

Figure A-2 shows two plots, one for each of two different values of the service rate p. 
Also, the two service rates have been relabeled as current NAS capacity and future NAS 
capacity. Note the vertical and horizontal line segments bounded by the two curves. The 
horizontal line segment represents the increase in demand at a given delay level made 
possible by the capacity increase. The vertical line segment represents a decrease in delay 
at a given demand level made possible by the capacity increase. If traffic demand is not 
increasing, a capacity increase should thus result in lower delay. If, however, demand is 
increasing, the system should observe a higher level of actual throughput, with delay 
levels possibly not significantly reduced. 

Whether or not an increase in capacity will lower delay levels or increase throughput is 
ultimately determined by the demand from the system users. 


23 



Demand X 




■> 


Figure A-l. Delay vs. demand in the simple queuing model. 
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Figure A-2. The basics: demand, capacity, and delay. 
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Appendix B. Airline Schedule Quality Issues in Air Traffic 
Management 

B.l Objectives 

A variety of measures of effectiveness have been employed for use in comparing 
alternative air-traffic -management (ATM) systems. An important one, which has 
received less attention than others, is airline schedule quality. This refers to schedule 
quality from a passenger point of view, and refers primarily to the ability of the airline 
schedules to offer sufficient travel opportunities in the places and at the times that they 
are needed. It is a long-term objective at Boeing to analyze and quantify the relationships 
that exist between air traffic system capacity and the resultant airline schedule quality. 
The preliminary analysis shown here is a first step in this direction and makes use of the 
following: 

• the Boeing airline schedule-generation capability developed for ATM concept 
evaluation, described in section B.2; and 

• a simple schedule quality metric developed to explore schedule quality issues 
under alternative air traffic system growth scenarios, described in section B.3. 

B.2 Overview of Future Airline Schedule-Generation Methodology 

An approach has been developed and exercised for creating future scheduled airline 
flights suitable for ATM concept analysis. Although this approach was developed and 
implemented prior to the work perfonned under this contract, it plays an important role in 
the schedule quality analysis shown in this section. The approach relies on: 

• projections from the Boeing Current Market Outlook (CMO); 

• an analysis of existing schedules as represented by the Official Airline Guide 
(OAG); and 

• algorithms for realizing the future schedules in an optimal way; a high-level 
view is shown in Figure B.2-1 and is explained in the following paragraph. 


For a future time period 

• Obtain CMO departures and 
available seat miles (ASMs) by airline 
and airplane type 

• Compute airline growth factors by 
comparing current / future total ASMs 

• Use airline growth factors to scale 
airport (AP) pair lift to the future 


AP pair lift 


Departures, ASMs, new lift 

► 


Today's OAG 

Schedule Visualization 

Analyze today's OAG 


Route map 
banks 

▼ Controls 

Schedule Generation 

Schedule CMO departures 
and ASMs to match future 
lift requirements 


New OAG 

▼ 

Figure B.2-1 Future airline schedule methodology. 
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The future schedules created thus far are for all flights in or out of the contiguous United 
States and are based upon the CMO traffic-forecasting process. This process utilizes an 
analysis of the current OAG and creates forecasts for various future years up to and 
including the year 2020. The model uses CMO forecasts of departures and available seat 
miles (ASMs) by equipment type and major airline. For this analysis, ten major U.S. 
airlines were explicitly considered with three aggregated carriers for cargo, foreign, and 
other U.S. carriers. The forecast showed a 77-percent increase in total departures from 
year 2000 to year 2020 with an average flight distance increase from 688 miles to 696 
miles, and with an approximate increase from 173 seats per mile flown to about 179 seats 
per mile flown. Thus, this baseline CMO forecast projects the vast majority of future 
demand to be met by increased frequencies, as opposed to being met by larger aircraft. 

Although the CMO forecast provides the total number of departures by equipment type 
and major airline, it does not indicate the assignment of these departures to particular 
nonstop airport pairs. In order to perform this assignment we must calculate the “lift” 

(i.e., number of seats) to be allocated to each such airport pair for the future time period. 
The set of nonstop airport pairs for an airline is called its route map. An analysis of 
existing schedules is first used to identify the segment route map for each major airline 
and, for each airport pair in the route map, the lift currently allocated to that pair. The 
future required lift for an airline airport pair is then calculated by projecting its current lift 
into the future by a factor computed from the ratio of the total CMO forecast ASMs 
divided by the total current ASMs for that airline. This assumes for each airline that there 
are no new nonstop airport pairs and that each existing pair grows at the same rate. The 
existing schedules are also analyzed to uncover the ba nk structure employed at each 
major airline hub. It is assumed that this bank structure will remain the same in future 
time periods, although the maximum bank sizes will be permitted to grow. 

When the future lift requirements (by airport pair and airline) are calculated, then an 
allocation process is undertaken to assign the forecast CMO airline/equipment departures 
to the individual airport pairs. The objective is to match as closely as possible both the 
required future lift on each airport pair and the forecast ASMs to be flown by each type 
of equipment for the given airline. This must be accomplished while respecting the 
design range limitations of the equipment. The departures (or arrivals) assigned to the 
hubs are then allocated to the airline banks in order to maximize schedule quality, using 
the Boeing Decision Window Model (DWM) as applied to the local markets, while 
respecting the maximum bank sizes and flow conservation. The last step in the process is 
to finalize the departure and arrival times for all flights (by any airline) while observing 
an airport specific limitation on the number of operations (takeoffs or landings) that may 
occur in any five-minute time window. Again, DWM is used to help select departure 
times based on schedule quality considerations. Depending on the scenario being 
evaluated the airports may be allowed to grow in their ability to process takeoffs and 
landings. When there is growth it is then assumed that the maximum bank sizes will grow 
correspondingly. 
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Although it is believed that this schedule-generation capability is unique in the industry, 
there are some areas for investigation and future development in order to improve upon 
the process. These areas include consideration of the following factors: 

• new nonstop airport pairs, including airports that are currently not in the OAG; 

• efficient, tail-routable schedules in order to better account for delay propagation 
through the airline networks; 

• redefined airline bank structures; 

• alternative CMO forecasts; and 

• schedule quality issues: the last of these areas is concerned with the fact that 
schedules created under alternative airspace constraints will not generally result in 
schedules of corresponding quality. The simple schedule quality indicator and 
preliminary analysis discussed in section B.3 represent a first look at this issue. 

B.3 A Simple Schedule Quality Indicator 

The analysis in section B.4 makes use of a simple schedule quality indicator. It is based 
upon time-of-day demand relationships developed and used in Boeing’s DWM of 
passenger path preference. Critical input to DWM are the arrival and departure “indices” 
or “weightings” illustrated in figure B.3-1. They show the relative desirability of 
departure times and arrival times as a function of the (local) time of day. In general, the 
desirability of flying at some time of day is a function of both the departure and arrival 
weightings. For this analysis we define “bad” departure hours and “bad” arrival hours to 
be the five worst hours of the day, respectively. Thus, as illustrated in figure B.3-1, the 
bad departure hours are 2-6, whereas the bad arrival hours are 1-5. We will label a flight 
as “bad” if either its departure or arrival time is bad. Then, for a given schedule, we will 
measure the fraction of seats on bad flights. Thus, our simple measure of schedule quality 
will actually be a measure of schedule “badness”. 

A metric as simple as this one has some issues. First, some (very) small fraction of 
travelers might actually prefer the bad departure or arrival times. Second, and much more 
important, not everybody would prefer just any “good” flight. In this context, good flights 
are anything not labeled bad. For example, a passenger might be satisfied by a morning 
flight but not an afternoon flight, even though both were labeled “good”. The important 
consequence is that if a schedule badness of 4-percent is calculated, for example, that 
does not mean that schedule “quality” is 96-percent. The intent is to capture an indication 
of schedule quality deterioration. That is, when a new schedule is created subject to air 
traffic capacity constraints, if an increase in schedule badness relative to today’s schedule 
is observed, it can be concluded that there is a likely deterioration in schedule quality. 
This is why the bad departure and arrival hours have been selected so that they are very 
bad. 
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Figure B.3-1 Departure/arrival weightings vs. time of day. 
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B.4 Preliminary Schedule Quality Analysis 

The objective of this preliminary analysis is to compute the simple schedule “badness” 
measure for alternative scenarios representing different assumptions concerning air traffic 
capacity. This capacity is expressed simply as a maximum number of arrival/departure 
operations for any five-minute period at an airport, based on the hourly airport capacity. 
Schedules are generated for some future time period subject to these constraints and are 
then evaluated. The following scenarios have been explored: 

OAG - year 2000 actual schedules as published by each airline 
BL - year 2000 generated schedules, using the methods described in section B.2 
NG - year 2020 generated schedules, no ATM capacity growth 
RG - year 2020 generated schedules, nominal runway growth 
AG - year 2020 generated schedules, Aviation System Technology Advanced 
Research (AvStar) program “goal” growth 
IG - year 2020 generated schedules, infinite ATM capacity 

For each of these scenarios various subsets of flights were evaluated: 

SYS - the entire (U.S. domestic) system of flights 

ORD - the subset of SYS involving ORD (Chicago O’Hare) flights 

ATL - the subset of SYS involving ATL (Atlanta) flights 

Baseline airport capacities were specified for the top 100 U.S. airports, as published in 
the Federal Aviation Administration (FAA) Capacity Baseline Report [8]. The remaining 
airports were given a maximum capacity of 60 operations per hour (or a maximum of 5 
operations per 5-minute period). The NG scenario assumes the same airport capacities (in 
2020) as are employed in the baseline scenario. The RG scenario assumes airport 
capacities are increased in accordance with forecast additional runways [8]. ORD was 
selected because it was an airport without forecast runway growth, whereas ATL was 
selected because an additional runway is planned to handle growth in this time frame. 

The AG scenario assumes 2-percent annual growth from the baseline over a 20-year 
period, according to the AvSTAR goal assumptions. This was applied to the top 100 
airports with no growth assumed for all remaining airports. 

Figure B.4-1 shows the fraction of airport capacity utilized as a function of hour of day 
for ORD. Results for this graph were aggregated to the one-hour level (versus the five- 
minute level) to smooth the results. The graph compares the BL and NG scenarios and 
indicates a shift of airport use into the early morning hours given the insufficient airport 
capacity, along with a filling-in of the times between traffic peaks. Results such as these 
motivated the creation of the schedule badness metric. Note, also, that the scheduled 
flying exhausts all available capacity during the daylight hours for the NG scenario. 
Under this scenario there will be little opportunity to recover during the day after 
disturbances such as weather or system outages that temporarily reduce capacity. 
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Figure B.4-1 Capacity utilization at ORD — 2020. 


Figure B.4-2 shows the schedule badness evaluations for the various cases under 
consideration. As expected, the OAG, BL, and IG cases are nearly the same in terms of 
this measure, particularly given the complexity of the schedule-generation process. There 
is, however, a distinct difference between these cases and the NG case. The significant 
increase in schedule badness for the NG case is a strong indicator of schedule quality 
deterioration due to insufficient airport capacity. The additional runways for the RG case 
(currently planned new runways) offer only slight relief for SYS and ATL and none for 
ORD. The AG case (AvSTAR goals) is also indistinguishable from the OAG, BL, and IG 
cases. This indicates that a 2-percent annual growth at the top 100 airports may be 
adequate to meet future needs. The authors believe, however, that more detailed analysis 
is needed to confirm that conclusion. Schedule badness was constructed so that when a 
problem is indicated it is a sure sign of schedule quality deterioration but “no indication” 
does not necessarily mean that schedule quality has been completely maintained. 
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Figure B.4-2 Preliminary results - schedule “badness” indicator. 


B.5 Forecasting Issues and Next Steps 

The primary issue in this analysis is that the schedule badness indicator is not a sufficient 
measure of schedule quality. A true measure of schedule quality would detennine the 
fraction of passenger demand that could be satisfied by the schedule. This index must 
account for, as a minimum: 

• origin-destination demand (connect paths); 

• time-of-day demand and schedule tolerance throughout the day; 

• passenger behavior (e.g. path-class choice); and 

• demand variability, airplane capacity, and revenue management. 

The schedule badness indicator captures only a small part of the time-of-day demand 
effects and the associated distribution of scheduled capacity, namely the undesirable 
hours in the middle of the night. It makes an implicit “local-demand-only” assumption in 
the sense that all seats on bad flights count as bad even though some of them will belong 
to connecting passengers. There are also some issues with the schedule-generation 
process, previously discussed in section B.2. Those issues are inherited in this analysis. 

Additionally, in the NG and RG cases, some flights forecast by the CMO could not be 
scheduled at all because of a lack of airport capacity. This is really an indication that the 
CMO forecast is inconsistent with the no-growth scenario. In reality, the airlines would 
adapt their scheduling and even business strategies to highly constrained airport 
capacities. In the future, alternative CMO forecasts should be developed to more closely 
match the various assumptions on airport or air traffic system capacity. It might also be 
advisable to explore more complex representations of airport capacity, particularly 
schemes allowing for a differentiation between peak and average capacities. 
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In tenns of schedule generation, Boeing's future activities will focus on at the creation of 
alternative CMO forecasts, development of efficient tail-routable schedules, 
consideration of alternative bank structures, and the development of origin-destination 
demand-driven airport-pair lift requirements. This last point would also include 
consideration of the policy questions leading to the inclusion of new “direct-fly” airport 
pairs. Boeing's efforts concerning schedule evaluation will focus on continued 
exploration and development of intermediate-level measures of schedule quality. The 
goal will be to develop an easy-to-understand and practical-to-compute approach 
applicable to large-scale analysis. 
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